Avoid expanding unreferenced struct plan fields - #9254
Conversation
Merging this PR will regress 1 benchmark
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
6f8023d to
25cf61e
Compare
f5f53a0 to
0f26545
Compare
0f26545 to
eafd835
Compare
eafd835 to
0d7be8a
Compare
0d7be8a to
37bf3de
Compare
37bf3de to
27052fc
Compare
27052fc to
4803863
Compare
4803863 to
67f0bde
Compare
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
67f0bde to
bfc1bf0
Compare
Summary
GetItem(root)expressions intact when partitioning a struct planSelect(root)into only the selected fieldsRoot cause
ExpressionStructRuleeagerly built a bound pack containing every struct field before it knewwhich fields the query referenced. On the 105-field ClickBench schema, every rule application
constructed 105
GetItemexpressions. Binding each node resolved its return dtype, repeatedlydeserializing lazy field dtypes even for a one-column query.
The expression is already bound, and the bound field annotator can partition direct field access
without expanding it. This change only synthesizes field accesses where
Selector a bare rootsemantically requires them.
Performance
The benchmark opens all 100 ClickBench Vortex shards, filters
AdvEngineID != 0, projectsAdvEngineID, warms planning once, and reports the median of 25 complete planning passes.BoundExpressionstackThis is a 66.6% planning reduction from the bound baseline and a 70.3% reduction from the original
stack. Execution is unchanged within run-to-run noise.
A matched Samply planning-only profile over 500 all-file passes reduced main-thread CPU from
8.22 s to 1.23 s. The previous hot stack through
BoundExpression::try_new,GetItem::return_dtype, and lazy dtype deserialization is absent after the change.Stack
This draft is stacked on #9244 and contains one signed commit.
Validation
RUSTC_WRAPPER= cargo nextest run -p vortex-layout -p vortex-scan-v2— 215 passedRUSTC_WRAPPER= cargo clippy --all-targets --all-features -- -D warningsRUSTC_WRAPPER= cargo test --doc -p vortex-layout -p vortex-scan-v2cargo +nightly fmt --all -- --checkgit diff --check